Hitherto, apparatuses in conformity with the system such as MPEG, etc. in which image information are handled as digital data to compress such image information by orthogonal transform such as Discrete Cosine Transform, etc. and motion prediction/compensation by utilizing redundancy specific to the image information with a view to performing efficient transmission/storage of information in this instance have been used in both information distribution (delivery) at broadcasting station, etc., and information reception in general homes.
Particularly, MPEG2 (ISO/IEC 13818-2) is defined as general purpose image encoding system, and is widely used at present for broad application of professional use purpose and consumer use purpose with the standard for covering both interlaced scanning image and sequential scanning image, and standard resolution image and high definition image. By using the MPEG2 compression system, e.g., in the case of interlaced scanning image of the standard resolution having 720 ×480 pixels, code quantity (bit rate) of 4 to 8 Mbps is allocated, and in the case of interlaced scanning image of the high resolution having 1920 ×1088 pixels, code quantity (bit rate) of 18 to 22 Mbps is allocated so that realization of high compression factor and satisfactory picture quality can be made.
The MPEG2 was mainly directed to high picture quality encoding adapted to broadcasting use purpose, but did not comply with the encoding system having code quantity (bit rate) lower than that of the MPEG1, i.e., higher compression factor. However, it is expected that need of such encoding system will be increased in future with popularization of portable (mobile) terminals, and standardization of MPEG4 encoding system was performed in correspondence therewith. In regard to the image encoding system, its standard was approved as the International Standard as ISO/IEC 14496-2 on December, 1998.
Further, in recent years, standardization of H.264 (ITU-TQ6/16 VCEG) is being developed with image encoding for television conference being as the initial or original object. It is known that while the H.264 is required to have a larger number of operation quantities with respect to encoding/decoding thereof as compared to the conventional encoding system such as MPEG2 or MPEG4, higher encoding efficiency can be realized. Moreover, at present, as a part of activity of the MPEG4, standardization in which functions which cannot be supported by the H.264 are also taken in with the H.264 being as base to realize higher encoding efficiency is being performed by JVT (Joint Video Team).
An image information encoding apparatus of a practical example of the encoding system in which standardization is performed by the JVT (hereinafter referred to as JVT Codec or H.264|MPEG-4 AVC) will be explained. FIG. 1 is a block diagram of the image information encoding apparatus adapted for realizing image compression by orthogonal transform such as Discrete Cosine Transform or Karhunen-Loeve Transform, etc. and motion compensation. As shown in FIG. 1, the image information encoding apparatus 100 comprises an A/D (Analogue/Digital) converter 101, an image sorting buffer 102, an adder 103, an orthogonal transform element 104, a quantizer 105, a reversible encoder 106, a storage buffer 107, an inverse-quantizer 108, an inverse-orthogonal transform element 109, a deblocking filter 110, a frame memory 111, a motion prediction/compensation element 112, an intra predictor 113, and a rate controller 114.
In FIG. 1, the A/D converter 101 converts an inputted image signal into a digital signal. The image sorting buffer 102 performs sorting operation of frames in accordance with GOP (Group of Pictures) structure of image compressed information outputted from the image information encoding apparatus 100. Here, in regard to image in which intra (intra-image) encoding is performed, the image sorting buffer 102 delivers image information of the entirety of frame to the orthogonal transform element 104. The orthogonal transform element 104 implements orthogonal transform such as Discrete Cosine Transform or Karhunen-Loeve Transform, etc. to the image information to deliver transform coefficients to the quantizer 105. The quantizer 105 implements quantization processing to the transform coefficients delivered from the orthogonal transform element 104.
The reversible encoder 106 implements reversible encoding such as variable length encoding or arithmetic encoding, etc. to the quantized transform coefficients to deliver the transform coefficients thus encoded to the storage buffer 107 to allow the storage buffer 107 to store those encoded transform coefficients. The encoded transform coefficients are outputted as image compressed information.
The behavior (operation) of the quantizer 105 is controlled by the rate controller 114. Moreover, the quantizer 105 delivers quantized transform coefficients to the inverse-quantizer 108. The inverse-quantizer 108 inverse-quantizes the transform coefficients. The inverse-orthoginal transform element 109 implements inverse orthogonal transform processing to the inverse-quantized transform coefficients to generate decoded image information. The deblocking filter 110 serves to remove block distortion from the decoded image information to deliver the decoded image information thus processed to the frame memory 111 to allow the frame memory 111 to store such decoded image information.
On the other hand, in regard to image in which inter (iner-image) encoding is performed, the image sorting buffer 102 delivers image information to a motion prediction/compensation element 112. The motion prediction/compensation element 112 takes out, from the frame memory 111, image information which is referred at the same time to implement motion prediction/compensation processing thereto to generate reference image information. The motion prediction/compensation element 112 delivers the reference image information to the adder 103. The adder 103 converts the refrence image information into difference signal between the reference image information and corresponding image information. Moreover, the motion compensation/prediction element 112 delivers, at the same time, motion vector information to the reversible encoder 106.
The reversible encoder 106 implements reversible encoding processing such as variable length encoding or arithmetic encoding, etc. to the motion vector information to form information which is inserted into header portion of image compressed information. It is to be noted that since other processing are similar to those of image compreseed information to which intra-encoding is implemented, the explanation thereof will be omitted.
Here, in the encoding system in which standardization is performed by the above-described JVT (hereinafter referred to as JVT Codec), there is employed the intra predictive encoding system of generating prediction image from pixels around block in performing intra encoding processing to encode difference between corresponding image and the prediction image. Namely, in regard to image in which intra-encoding is performed, prediction image is generated from pixel value in which encoding processing has been already completed, which is in the vicinity of pixel block to be encoded, so that difference with respect to the prediction image is encoded. The inverse-quantizer 108 and the inverse-orthogonal transform element 109 respectively inverse-quantize and inverse-orthogonally transform intra-encoded pixels. The adder 103 adds output of the inverse-orthogonal transform element 109 and prediction image used when corresponding pixel block is encoded to deliver the added value thereof to the frame memory 111 to allow the frame memory 111 to store the added value thus obtained. In the case of pixel block to be intra-encoded, an intra predictor 113 reads out neighboring pixels stored in the frame memory 111 in which encoding processing has been already completed to generate prediction image. At this time, also with respect to the intra prediction mode used for generation of prediction image, at the reversible encoder 106, reversible encoding processing is implemented thereto to output the prediction image thus obtained in the state included in image compressed information.
Subsequently, outline of the configuration of an image information decoding apparatus corresponding to the above-described image information encoding apparatus 100 is shown in FIG. 2. The image information decoding apparatus 120 comprises, as shown in FIG. 2, a storage buffer 121, a reversible decoder 122, an inverse-quantizer 123, an inverse-orthogonal transform element 124, an adder 125, an image sorting buffer 126, a D/A (Digital/Analogue) converter 127, a motion prediction/compensation element 128, a frame memory 129, and an intra-predictor 130.
In FIG. 2, the storage buffer 121 temporarily stores inputted image compressed information thereafter to transfer that image compressed information to the reversible decoder 122. The reversible decoder 122 implements processing such as variable length decoding or arithmetic decoding, etc. to the image compressed information on the basis of the determined format for image compressed information to deliver quantized transform coefficients to the inverse-quantizer 123. Moreover, in the case where corresponding frame is inter-encoded frame, the reversible decoder 122 also decodes motion vector information stored in the header portion of image compressed information to deliver the decoded motion vector information thus obtained to the motion prediction/compensation element 128.
The inverse-quantizer 123 inverse-quantizes the quantized transform coefficients which have been delivered from the reversible decoder 122 to deliver the transform coefficients thus processed to the inverse-orthogonal transform element 124. The inverse-orthogonal transform element 124 implements inverse-orthogonal transform such as Inverse Discrete Cosine Transform or Inverse Karhunen-Loeve Transform, etc. to the transform coefficients on the basis of the determined format for image compressed information.
Here, in the case where corresponding frame is intra-encoded frame, image information to which inverse-orthogonal transform processing has been implemented is stored into the image sorting buffer 126. After undergone D/A transform processing at the D/A converter 127, the image information thus processed is outputted.
On the other hand, in the case where corresponding frame is inter-encoded frame, the motion prediction/compensation element 128 generates reference image on the basis of motion vector information to which reversible decoding has been implemented and image information stored in the frame memory 129 to deliver the reference image thus generated to the adder 125. The adder 125 synthesizes the reference image and an output of the inverse-orthogonal transform element 124. It is to be noted that since other processing are similar to those of intra-encoded frame, the explanation thereof will be omitted.
Since the intra-predictive encoding system is employed in the JVT Codec in this example, in the case where corresponding frame is intra-encoded frame, the intra predictor 130 reads out image from the frame memory 129 to generate prediction image in accordance with the intra prediction mode to which reversible decoding processing has been implemented at the reversible decoder 122. The adder 125 adds an output of the inverse-orthogonal transform element 124 and the prediction image.
The image information encoding apparatus 100 and the image information decoding apparatus 120 which have been explained above are described in the Japanese Patent Application Laid Open No. 2001-199818 publication, and the Japanese Patent Application Laid Open No. 2002-20953 publication.
Meanwhile, at the inverse-orthogonal transform element 124 shown in FIG. 2, matrix calculations of 8×8/4×4 of two times are performed in length and breadth directions, e.g., in units of block of 8×8/4×4 pixels with respect to transform coefficients to implement inverse orthogonal processing. However, at the decoding processing, in order to perform inverse-transform processing of all blocks in which an image is divided into 8×8/4×4 pixels, calculation quantity would become vast (large). Since there is no change of value also after matrix processing (transformed values are all zero) with respect to block in which any coefficient does not exist (pixel values are all zero), wasteful calculation would be performed. In addition, also in the case where only DC component of block exists, or deviation of coefficients exists within block, there are instances where efficiency is not good for the purpose of performing two times of matrix calculations of 8×8/4×4.